Fully Variational Noise-Contrastive Estimation

By using the underlying theory of proper scoring rules, we design a family noise-contrastive estimation (NCE) methods that are tractable for latent variable models. Both terms in NCE loss, one data samples and noise samples, can be lower-bounded as variational Bayes, therefore call this losses fully estimation. Variational autoencoders particular example also understood separating real from synthetic an appropriate classification loss. We further discuss other instances objectives indicate differences their empirical behavior.